Combining Methods: Web Analytics and User Research presentation delivered at the eMetrics Marketing Optimization Summit in Stockholm, Sweden.

1. Combining Methods Web Analytics and User Research Martijn Klompenhouwer & Adam Cox eMetrics - September 28 th , 2010 Who are we..? Web Analytics User Research Why you should combine methods Practical examples Why you should try it too

2. About User Intelligence

3. Martijn Klompenhouwer UX researcher 11 years UX experience 8 years at User Intelligence

4. Adam Cox Web Analyst 7 years UX / Analyst experience 3 years at User Intelligence

7. About Web Analytics

9. What does the definition mean? Where are your users coming from? What are they doing? Where and when are they leaving?

10. Some examples of Web Analytic measurements Referrals Visitors, visits and page views Popular pages Bounce rates Funnel analysis Path Analysis

11. Issues with Web Analytics  Data often perceived as boring statistics  Implementation could miss relevant metrics  Reports: too detailed and analytic for audience

12. Web Analytics is quantitative in nature To get insights, interpretation of the numbers is needed

13. About User Research

14. Some examples of User Research methods Field observations Personas Diary studies Card sorting Expert Review Interviews Focus groups User Testing

15. Issues with User Research methods  Data usually from small numbers  Most methods take a snapshot in time  Difficult to capture some behavior  Setting sometimes artificial (e.g. lab test)

16. User Research is qualitative in nature You get the ‘why’, but not the ‘big numbers’…

17. So, why should you combine these two methods?  User Research findings can help interpret web data  Web data can help focus the User Research  More certainty of findings (based on two sources)

18.  The methods complement each other Quantitative vs. Qualitative

19. Let’s clarify with some examples…

20. Example 1: Unexpected landing page Entry point One page generated more traffic than the homepage User expectations Goals of the users not being met Redesign page Without WA, this issue would have gone unnoticed

21. Example 2: Usability test Google scenario Testing outcomes of a common scenario User journeys Homepage was not the main entry point Test scenarios Creating realistic test scenarios: how is site used now..?

22. Example 3: Use of Advanced features Popular settings? Which options used and are there any patterns or issues? Collect data Decided not to use too much test time: wait for proper measurements Not measured Use of feature was known, but no data on the use of settings

23. Example 4: Unintended user flows Alternative route Internal search engine often not used, lot of direct traffic. Dead-end page Page offered visitors no reasons to stay or explore site further Redesign page Options were added to draw visitors ‘back’ into the main site

24. Example 5: Researching abandonment rates Funnel analysis Sales funnel analysis revealed where users left the flow Target test tasks Knowledge enabled us to concentrate test on those steps More focus Finding reasons for abandonment and looking for solutions

25. Example 6: Validating findings End the discussion Discussion changed from: “Is that an issue?” to: “Let’s solve it!” Impact analysis Quantified issue using data of thousands of visitors Big issue? Only 2 out of 10 test participants had this problem

26. Combining findings Single deliverable Combined approach also means combining your findings… Two sources Present the data from the two methods together Single message No need to compare two different reports: clear action points

27. Examples of combined reports (1/3)

28. Examples of combined reports (2/3)

29. Examples of combined reports (3/3)

30. Some additional examples… Effectiveness of segmentation Verifying user feedback Explaining drop-out rates Identifying target audience

31. Why you should try it too  Combining methods works! You can help sell the story by backing it up with solid data  Integrated findings: no conflicting recommendations  Useful in different stages of a project (Research, Design, Optimize)  The combination works both ways! Web Analyst and User Researcher benefit from each other  Many User Research methods will profit from WA data

33. What we want you to take away…  Find out who and where the UX team is  Approach them!..... Just try it! No more excuses ;-)  Web data will help in different stages of a project  Integrate Web Analytics into the UX process and methods  Interpreting the data together will lead to better insights  Understand each others strengths and needs Provide data of right level of detail that can easily be used

34. user intelligence Amsterdam office www.userintelligence.com Thank you! Martijn Klompenhouwer [email_address] Adam Cox [email_address]